Method and apparatus for partial correction of images

ABSTRACT

An image capture device may include an image sensor, a processor, and memory. The image sensor may be configured to obtain an image. The processor may be configured to: generate a grid on the image forming tiles; determine a fringing level of each vertex of the vertices; sort all of the tiles based on the fringing level of each tile so that the tiles are sorted in a descending order from the tile with a highest of the fringing levels to the tile with a lowest of the fringing levels; and apply a fringing compensation to a subset of the sorted tiles to obtain a processed image. The memory may be configured to store the processed image.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation of U.S. patent application Ser. No.17/000,822, filed Aug. 24, 2020, which claims priority to and thebenefit of U.S. Provisional Patent Application No. 62/900,945, filedSep. 16, 2019, the entire disclosures of which are hereby incorporatedby reference.

TECHNICAL FIELD

This disclosure relates to image capture devices and partial correctionof images.

BACKGROUND

High performance video recording requires high pixel rate, complexalgorithms, and high power consumption that cannot be delivered bytypical cameras. These typical cameras include central processing units(CPU)s that cannot handle the complex algorithms required for highperformance video recording, and attempting to implement the complexalgorithms in such CPUs results in a video recording with poor imagequality.

SUMMARY

Disclosed herein are implementations of partial correction of images. Inan aspect, an image capture device may include an image sensor, aprocessor, and a memory. The image sensor may be configured to obtain animage. The processor may be configured to generate a grid on the image.The grid may include one or more vertices. The one or more vertices maybe used to form tiles. The processor may be configured to determine aflare level of each vertex. The processor may be configured to assign amaximum flare level for each tile of the image. The processor may beconfigured to sort the tiles. The tiles may be sorted based on themaximum flare level of each tile. The processor may be configured toapply a flare compensation to a subset of the sorted tiles to obtain aprocessed image. The processed image may have reduced flare artifacts orno flare artifacts. The processed image may be stored in the memory.

In another aspect, a method may include obtaining an image. The methodmay include generating a grid on the image. The grid may include one ormore vertices. The one or more vertices may be used to form tiles. Themethod may include determining a flare level on the one or morevertices. The method may include applying a flare compensation to asubset of the tiles to obtain a processed image.

In another aspect, an image capture device may include an image sensor,a processor, and a memory. The image sensor may be configured to obtainan image. The processor may be configured to determine a thumbnailimage. The thumbnail image may be based on the image. The thumbnailimage may include one or more thumbnail tiles. The processor may beconfigured to determine a contrast value of each thumbnail tile. Theprocessor may be configured to sort the one or more thumbnail tiles. Theone or more thumbnail tiles may be sorted based on the contrast value ofeach thumbnail tile. The processor may be configured to apply acompensation value to a subset of the sorted thumbnail tiles to obtain aprocessed image. The memory may be configured to store the processedimage.

In yet another aspect, an image capture device may include an imagesensor, a processor, and memory. The image sensor may be configured toobtain an image. The processor may be configured to: generate a grid onthe image forming tiles; determine a fringing level of each vertex ofthe vertices; sort all of the tiles based on the fringing level of eachtile so that the tiles are sorted in a descending order from the tilewith a highest of the fringing levels to the tile with a lowest of thefringing levels; and apply a fringing compensation to a subset of thesorted tiles to obtain a processed image. The memory may be configuredto store the processed image.

One aspect provides a method comprising: obtaining an image. Thengenerating a grid on the image, wherein the grid comprises vertices toform tiles. Determining fringing levels for the vertices. Finally,applying a fringing compensation to a subset of the tiles based on thefringing levels to obtain a processed image, wherein applying thefringing compensation includes forcing a zero-fringing compensation atan edge between a first tile and a second tile of the tiles.

Another aspect provides an image capture device comprising: an imagesensor, a processor, and memory. The image sensor is configured toobtain an image. The processor is configured to: determine a thumbnailimage based on the image, wherein the thumbnail image comprisesthumbnail tiles; determine a saturation value of each thumbnail tile;sort all of the thumbnail tiles based on the saturation value of eachthumbnail tile in an ascending order from the thumbnail tile with alowest saturation value to the thumbnail tile with a highest saturationvalue; and apply a correction to a subset of the sorted thumbnail tilesto obtain a processed image. The memory is configured to store theprocessed image.

BRIEF DESCRIPTION OF THE DRAWINGS

The disclosure is best understood from the following detaileddescription when read in conjunction with the accompanying drawings. Itis emphasized that, according to common practice, the various featuresof the drawings are not to-scale. On the contrary, the dimensions of thevarious features are arbitrarily expanded or reduced for clarity.

FIGS. 1A-B are isometric views of an example of an image capture device.

FIGS. 2A-B are isometric views of another example of an image capturedevice.

FIG. 2C is a cross-sectional view of the image capture device of FIGS.2A-B.

FIG. 2D is a partial cross-sectional view of the image capture device ofFIG. 2C.

FIG. 3 is a block diagram of electronic components of an image capturedevice.

FIG. 4 is a flow diagram of an example of a method for flarecompensation in accordance with embodiments of this disclosure.

FIG. 5A is a block diagram of a tiled image in accordance withembodiments of this disclosure.

FIG. 5B is a block diagram of the tiled image of FIG. 5A showingselected tiles for flare compensation.

FIG. 6 is a flow diagram of an example of a method for contrastcompensation in accordance with embodiments of this disclosure.

DETAILED DESCRIPTION

Implementations disclosed herein may include processing a portion of animage instead of the entire image to increase processing speed andefficiency without reducing image quality. Since some image artifactsonly affect a small portion of the image, the implementations describedherein determine which pixels of the image would benefit the most fromprocessing without sacrificing image quality. The implementationsdescribed herein may be applied to any type of image correction, forexample, flare compensation, blue-fringing correction, and local tonemapping (LTM).

FIGS. 1A-B are isometric views of an example of an image capture device100. The image capture device 100 may include a body 102, a lens 104structured on a front surface of the body 102, various indicators on thefront surface of the body 102 (such as light-emitting diodes (LEDs),displays, and the like), various input mechanisms (such as buttons,switches, and/or touch-screens), and electronics (such as imagingelectronics, power electronics, etc.) internal to the body 102 forcapturing images via the lens 104 and/or performing other functions. Thelens 104 is configured to receive light incident upon the lens 104 andto direct received light onto an image sensor internal to the body 102.The image capture device 100 may be configured to capture images andvideo and to store captured images and video for subsequent display orplayback.

The image capture device 100 may include an LED or another form ofindicator 106 to indicate a status of the image capture device 100 and aliquid-crystal display (LCD) or other form of a display 108 to showstatus information such as battery life, camera mode, elapsed time, andthe like. The image capture device 100 may also include a mode button110 and a shutter button 112 that are configured to allow a user of theimage capture device 100 to interact with the image capture device 100.For example, the mode button 110 and the shutter button 112 may be usedto turn the image capture device 100 on and off, scroll through modesand settings, and select modes and change settings. The image capturedevice 100 may include additional buttons or interfaces (not shown) tosupport and/or control additional functionality.

The image capture device 100 may include a door 114 coupled to the body102, for example, using a hinge mechanism 116. The door 114 may besecured to the body 102 using a latch mechanism 118 that releasablyengages the body 102 at a position generally opposite the hingemechanism 116. The door 114 may also include a seal 120 and a batteryinterface 122. When the door 114 is an open position, access is providedto an input-output (I/O) interface 124 for connecting to orcommunicating with external devices as described below and to a batteryreceptacle 126 for placement and replacement of a battery (not shown).The battery receptacle 126 includes operative connections (not shown)for power transfer between the battery and the image capture device 100.When the door 114 is in a closed position, the seal 120 engages a flange(not shown) or other interface to provide an environmental seal, and thebattery interface 122 engages the battery to secure the battery in thebattery receptacle 126. The door 114 can also have a removed position(not shown) where the entire door 114 is separated from the imagecapture device 100, that is, where both the hinge mechanism 116 and thelatch mechanism 118 are decoupled from the body 102 to allow the door114 to be removed from the image capture device 100.

The image capture device 100 may include a microphone 128 on a frontsurface and another microphone 130 on a side surface. The image capturedevice 100 may include other microphones on other surfaces (not shown).The microphones 128, 130 may be configured to receive and record audiosignals in conjunction with recording video or separate from recordingof video. The image capture device 100 may include a speaker 132 on abottom surface of the image capture device 100. The image capture device100 may include other speakers on other surfaces (not shown). Thespeaker 132 may be configured to play back recorded audio or emit soundsassociated with notifications.

A front surface of the image capture device 100 may include a drainagechannel 134. A bottom surface of the image capture device 100 mayinclude an interconnect mechanism 136 for connecting the image capturedevice 100 to a handle grip or other securing device. In the exampleshown in FIG. 1B, the interconnect mechanism 136 includes foldingprotrusions configured to move between a nested or collapsed position asshown and an extended or open position (not shown) that facilitatescoupling of the protrusions to mating protrusions of other devices suchas handle grips, mounts, clips, or like devices.

The image capture device 100 may include an interactive display 138 thatallows for interaction with the image capture device 100 whilesimultaneously displaying information on a surface of the image capturedevice 100.

The image capture device 100 of FIGS. 1A-B includes an exterior thatencompasses and protects internal electronics. In the present example,the exterior includes six surfaces (i.e. a front face, a left face, aright face, a back face, a top face, and a bottom face) that form arectangular cuboid. Furthermore, both the front and rear surfaces of theimage capture device 100 are rectangular. In other embodiments, theexterior may have a different shape. The image capture device 100 may bemade of a rigid material such as plastic, aluminum, steel, orfiberglass. The image capture device 100 may include features other thanthose described here. For example, the image capture device 100 mayinclude additional buttons or different interface features, such asinterchangeable lenses, cold shoes, and hot shoes that can addfunctional features to the image capture device 100.

The image capture device 100 may include various types of image sensors,such as charge-coupled device (CCD) sensors, active pixel sensors (APS),complementary metal-oxide-semiconductor (CMOS) sensors, N-typemetal-oxide-semiconductor (NMOS) sensors, and/or any other image sensoror combination of image sensors.

Although not illustrated, in various embodiments, the image capturedevice 100 may include other additional electrical components (e.g., animage processor, camera system-on-chip (SoC), etc.), which may beincluded on one or more circuit boards within the body 102 of the imagecapture device 100.

The image capture device 100 may interface with or communicate with anexternal device, such as an external user interface device (not shown),via a wired or wireless computing communication link (e.g., the I/Ointerface 124). Any number of computing communication links may be used.The computing communication link may be a direct computing communicationlink or an indirect computing communication link, such as a linkincluding another device or a network, such as the internet, may beused.

In some implementations, the computing communication link may be a Wi-Filink, an infrared link, a Bluetooth (BT) link, a cellular link, a ZigBeelink, a near field communications (NFC) link, such as an ISO/IEC 20643protocol link, an Advanced Network Technology interoperability (ANT+)link, and/or any other wireless communications link or combination oflinks.

In some implementations, the computing communication link may be an HDMIlink, a USB link, a digital video interface link, a display portinterface link, such as a Video Electronics Standards Association (VESA)digital display interface link, an Ethernet link, a Thunderbolt link,and/or other wired computing communication link.

The image capture device 100 may transmit images, such as panoramicimages, or portions thereof, to the external user interface device viathe computing communication link, and the external user interface devicemay store, process, display, or a combination thereof the panoramicimages.

The external user interface device may be a computing device, such as asmartphone, a tablet computer, a phablet, a smart watch, a portablecomputer, personal computing device, and/or another device orcombination of devices configured to receive user input, communicateinformation with the image capture device 100 via the computingcommunication link, or receive user input and communicate informationwith the image capture device 100 via the computing communication link.

The external user interface device may display, or otherwise present,content, such as images or video, acquired by the image capture device100. For example, a display of the external user interface device may bea viewport into the three-dimensional space represented by the panoramicimages or video captured or created by the image capture device 100.

The external user interface device may communicate information, such asmetadata, to the image capture device 100. For example, the externaluser interface device may send orientation information of the externaluser interface device with respect to a defined coordinate system to theimage capture device 100, such that the image capture device 100 maydetermine an orientation of the external user interface device relativeto the image capture device 100.

Based on the determined orientation, the image capture device 100 mayidentify a portion of the panoramic images or video captured by theimage capture device 100 for the image capture device 100 to send to theexternal user interface device for presentation as the viewport. In someimplementations, based on the determined orientation, the image capturedevice 100 may determine the location of the external user interfacedevice and/or the dimensions for viewing of a portion of the panoramicimages or video.

The external user interface device may implement or execute one or moreapplications to manage or control the image capture device 100. Forexample, the external user interface device may include an applicationfor controlling camera configuration, video acquisition, video display,or any other configurable or controllable aspect of the image capturedevice 100.

The user interface device, such as via an application, may generate andshare, such as via a cloud-based or social media service, one or moreimages, or short video clips, such as in response to user input. In someimplementations, the external user interface device, such as via anapplication, may remotely control the image capture device 100 such asin response to user input.

The external user interface device, such as via an application, maydisplay unprocessed or minimally processed images or video captured bythe image capture device 100 contemporaneously with capturing the imagesor video by the image capture device 100, such as for shot framing orlive preview, and which may be performed in response to user input. Insome implementations, the external user interface device, such as via anapplication, may mark one or more key moments contemporaneously withcapturing the images or video by the image capture device 100, such aswith a tag or highlight in response to a user input or user gesture.

The external user interface device, such as via an application, maydisplay or otherwise present marks or tags associated with images orvideo, such as in response to user input. For example, marks may bepresented in a camera roll application for location review and/orplayback of video highlights.

The external user interface device, such as via an application, maywirelessly control camera software, hardware, or both. For example, theexternal user interface device may include a web-based graphicalinterface accessible by a user for selecting a live or previouslyrecorded video stream from the image capture device 100 for display onthe external user interface device.

The external user interface device may receive information indicating auser setting, such as an image resolution setting (e.g., 3840 pixels by2160 pixels), a frame rate setting (e.g., 60 frames per second (fps)), alocation setting, and/or a context setting, which may indicate anactivity, such as mountain biking, in response to user input, and maycommunicate the settings, or related information, to the image capturedevice 100.

The image capture device 100 may be used to implement some or all of themethods described in this disclosure, such as the method 400 describedin FIG. 4 .

FIGS. 2A-B illustrate another example of an image capture device 200.The image capture device 200 includes a body 202 and two camera lenses204 and 206 disposed on opposing surfaces of the body 202, for example,in a back-to-back configuration, Janus configuration, or offset Janusconfiguration. The body 202 of the image capture device 200 may be madeof a rigid material such as plastic, aluminum, steel, or fiberglass.

The image capture device 200 includes various indicators on the front ofthe surface of the body 202 (such as LEDs, displays, and the like),various input mechanisms (such as buttons, switches, and touch-screenmechanisms), and electronics (e.g., imaging electronics, powerelectronics, etc.) internal to the body 202 that are configured tosupport image capture via the two camera lenses 204 and 206 and/orperform other imaging functions.

The image capture device 200 includes various indicators, for example,LEDs 208, 210 to indicate a status of the image capture device 100. Theimage capture device 200 may include a mode button 212 and a shutterbutton 214 configured to allow a user of the image capture device 200 tointeract with the image capture device 200, to turn the image capturedevice 200 on, and to otherwise configure the operating mode of theimage capture device 200. It should be appreciated, however, that, inalternate embodiments, the image capture device 200 may includeadditional buttons or inputs to support and/or control additionalfunctionality.

The image capture device 200 may include an interconnect mechanism 216for connecting the image capture device 200 to a handle grip or othersecuring device. In the example shown in FIGS. 2A and 2B, theinterconnect mechanism 216 includes folding protrusions configured tomove between a nested or collapsed position (not shown) and an extendedor open position as shown that facilitates coupling of the protrusionsto mating protrusions of other devices such as handle grips, mounts,clips, or like devices.

The image capture device 200 may include audio components 218, 220, 222such as microphones configured to receive and record audio signals(e.g., voice or other audio commands) in conjunction with recordingvideo. The audio component 218, 220, 222 can also be configured to playback audio signals or provide notifications or alerts, for example,using speakers. Placement of the audio components 218, 220, 222 may beon one or more of several surfaces of the image capture device 200. Inthe example of FIGS. 2A and 2B, the image capture device 200 includesthree audio components 218, 220, 222, with the audio component 218 on afront surface, the audio component 220 on a side surface, and the audiocomponent 222 on a back surface of the image capture device 200. Othernumbers and configurations for the audio components are also possible.

The image capture device 200 may include an interactive display 224 thatallows for interaction with the image capture device 200 whilesimultaneously displaying information on a surface of the image capturedevice 200. The interactive display 224 may include an I/O interface,receive touch inputs, display image information during video capture,and/or provide status information to a user. The status informationprovided by the interactive display 224 may include battery power level,memory card capacity, time elapsed for a recorded video, etc.

The image capture device 200 may include a release mechanism 225 thatreceives a user input to in order to change a position of a door (notshown) of the image capture device 200. The release mechanism 225 may beused to open the door (not shown) in order to access a battery, abattery receptacle, an I/O interface, a memory card interface, etc. (notshown) that are similar to components described in respect to the imagecapture device 100 of FIGS. 1A and 1B.

In some embodiments, the image capture device 200 described hereinincludes features other than those described. For example, instead ofthe I/O interface and the interactive display 224, the image capturedevice 200 may include additional interfaces or different interfacefeatures. For example, the image capture device 200 may includeadditional buttons or different interface features, such asinterchangeable lenses, cold shoes, and hot shoes that can addfunctional features to the image capture device 200.

FIG. 2C is a top view of the image capture device 200 of FIGS. 2A-B andFIG. 2D is a partial cross-sectional view of the image capture device200 of FIG. 2C. The image capture device 200 is configured to capturespherical images, and accordingly, includes a first image capture device226 and a second image capture device 228. The first image capturedevice 226 defines a first field-of-view 230 and includes the lens 204that receives and directs light onto a first image sensor 232.Similarly, the second image capture device 228 defines a secondfield-of-view 234 and includes the lens 206 that receives and directslight onto a second image sensor 236. To facilitate the capture ofspherical images, the image capture devices 226 and 228 (and relatedcomponents) may be arranged in a back-to-back (Janus) configuration suchthat the lenses 204, 206 face in generally opposite directions.

The fields-of-view 230, 234 of the lenses 204, 206 are shown above andbelow boundaries 238, 240 indicated in dotted line. Behind the firstlens 204, the first image sensor 232 may capture a firsthyper-hemispherical image plane from light entering the first lens 204,and behind the second lens 206, the second image sensor 236 may capturea second hyper-hemispherical image plane from light entering the secondlens 206.

One or more areas, such as blind spots 242, 244 may be outside of thefields-of-view 230, 234 of the lenses 204, 206 so as to define a “deadzone.” In the dead zone, light may be obscured from the lenses 204, 206and the corresponding image sensors 232, 236, and content in the blindspots 242, 244 may be omitted from capture. In some implementations, theimage capture devices 226, 228 may be configured to minimize the blindspots 242, 244.

The fields-of-view 230, 234 may overlap. Stitch points 246, 248 proximalto the image capture device 200, that is, locations at which thefields-of-view 230, 234 overlap, may be referred to herein as overlappoints or stitch points. Content captured by the respective lenses 204,206 that is distal to the stitch points 246, 248 may overlap.

Images contemporaneously captured by the respective image sensors 232,236 may be combined to form a combined image. Generating a combinedimage may include correlating the overlapping regions captured by therespective image sensors 232, 236, aligning the captured fields-of-view230, 234, and stitching the images together to form a cohesive combinedimage.

A slight change in the alignment, such as position and/or tilt, of thelenses 204, 206, the image sensors 232, 236, or both, may change therelative positions of their respective fields-of-view 230, 234 and thelocations of the stitch points 246, 248. A change in alignment mayaffect the size of the blind spots 242, 244, which may include changingthe size of the blind spots 242, 244 unequally.

Incomplete or inaccurate information indicating the alignment of theimage capture devices 226, 228, such as the locations of the stitchpoints 246, 248, may decrease the accuracy, efficiency, or both ofgenerating a combined image. In some implementations, the image capturedevice 200 may maintain information indicating the location andorientation of the lenses 204, 206 and the image sensors 232, 236 suchthat the fields-of-view 230, 234, the stitch points 246, 248, or bothmay be accurately determined; the maintained information may improve theaccuracy, efficiency, or both of generating a combined image.

The lenses 204, 206 may be laterally offset from each other, may beoff-center from a central axis of the image capture device 200, or maybe laterally offset and off-center from the central axis. As compared toimage capture devices with back-to-back lenses, such as lenses alignedalong the same axis, image capture devices including laterally offsetlenses may include substantially reduced thickness relative to thelengths of the lens barrels securing the lenses. For example, theoverall thickness of the image capture device 200 may be close to thelength of a single lens barrel as opposed to twice the length of asingle lens barrel as in a back-to-back lens configuration. Reducing thelateral distance between the lenses 204, 206 may improve the overlap inthe fields-of-view 230, 234. In another embodiment (not shown), thelenses 204, 206 may be aligned along a common imaging axis.

Images or frames captured by the image capture devices 226, 228 may becombined, merged, or stitched together to produce a combined image, suchas a spherical or panoramic image, which may be an equirectangularplanar image. In some implementations, generating a combined image mayinclude use of techniques including noise reduction, tone mapping, whitebalancing, or other image correction. In some implementations, pixelsalong the stitch boundary may be matched accurately to minimize boundarydiscontinuities.

The image capture device 200 may be used to implement some or all of themethods described in this disclosure, such as the method 400 describedin FIG. 4 .

FIG. 3 is a block diagram of electronic components in an image capturedevice 300. The image capture device 300 may be a single-lens imagecapture device, a multi-lens image capture device, or variationsthereof, including an image capture device with multiple capabilitiessuch as use of interchangeable integrated sensor lens assemblies. Thedescription of the image capture device 300 is also applicable to theimage capture devices 100, 200 of FIGS. 1A-B and 2A-D.

The image capture device 300 includes a body 302 which includeselectronic components such as capture components 310, a processingapparatus 320, data interface components 330, movement sensors 340,power components 350, and/or user interface components 360.

The capture components 310 include one or more image sensors 312 forcapturing images and one or more microphones 314 for capturing audio.

The image sensor(s) 312 is configured to detect light of a certainspectrum (e.g., the visible spectrum or the infrared spectrum) andconvey information constituting an image as electrical signals (e.g.,analog or digital signals). The image sensor(s) 312 detects lightincident through a lens coupled or connected to the body 302. The imagesensor(s) 312 may be any suitable type of image sensor, such as acharge-coupled device (CCD) sensor, active pixel sensor (APS),complementary metal-oxide-semiconductor (CMOS) sensor, N-typemetal-oxide-semiconductor (NMOS) sensor, and/or any other image sensoror combination of image sensors. Image signals from the image sensor(s)312 may be passed to other electronic components of the image capturedevice 300 via a bus 380, such as to the processing apparatus 320. Insome implementations, the image sensor(s) 312 includes adigital-to-analog converter. A multi-lens variation of the image capturedevice 300 can include multiple image sensors 312.

The microphone(s) 314 is configured to detect sound, which may berecorded in conjunction with capturing images to form a video. Themicrophone(s) 314 may also detect sound in order to receive audiblecommands to control the image capture device 300.

The processing apparatus 320 may be configured to perform image signalprocessing (e.g., filtering, tone mapping, stitching, and/or encoding)to generate output images based on image data from the image sensor(s)312. The processing apparatus 320 may include one or more processorshaving single or multiple processing cores. In some implementations, theprocessing apparatus 320 may include an application specific integratedcircuit (ASIC). For example, the processing apparatus 320 may include acustom image signal processor. The processing apparatus 320 may exchangedata (e.g., image data) with other components of the image capturedevice 300, such as the image sensor(s) 312, via the bus 380.

The processing apparatus 320 may include memory, such as a random-accessmemory (RAM) device, flash memory, or another suitable type of storagedevice, such as a non-transitory computer-readable memory. The memory ofthe processing apparatus 320 may include executable instructions anddata that can be accessed by one or more processors of the processingapparatus 320. For example, the processing apparatus 320 may include oneor more dynamic random-access memory (DRAM) modules, such as double datarate synchronous dynamic random-access memory (DDR SDRAM). In someimplementations, the processing apparatus 320 may include a digitalsignal processor (DSP). More than one processing apparatus may also bepresent or associated with the image capture device 300.

The data interface components 330 enable communication between the imagecapture device 300 and other electronic devices, such as a remotecontrol, a smartphone, a tablet computer, a laptop computer, a desktopcomputer, or a storage device. For example, the data interfacecomponents 330 may be used to receive commands to operate the imagecapture device 300, transfer image data to other electronic devices,and/or transfer other signals or information to and from the imagecapture device 300. The data interface components 330 may be configuredfor wired and/or wireless communication. For example, the data interfacecomponents 330 may include an I/O interface 332 that provides wiredcommunication for the image capture device, which may be a USB interface(e.g., USB type-C), a high-definition multimedia interface (HDMI), or aFireWire interface. The data interface components 330 may include awireless data interface 334 that provides wireless communication for theimage capture device 300, such as a Bluetooth interface, a ZigBeeinterface, and/or a Wi-Fi interface. The data interface components 330may include a storage interface 336, such as a memory card slotconfigured to receive and operatively couple to a storage device (e.g.,a memory card) for data transfer with the image capture device 300(e.g., for storing captured images and/or recorded audio and video).

The movement sensors 340 may detect the position and movement of theimage capture device 300. The movement sensors 340 may include aposition sensor 342, an accelerometer 344, or a gyroscope 346. Theposition sensor 342, such as a global positioning system (GPS) sensor,is used to determine a position of the image capture device 300. Theaccelerometer 344, such as a three-axis accelerometer, measures linearmotion (e.g., linear acceleration) of the image capture device 300. Thegyroscope 346, such as a three-axis gyroscope, measures rotationalmotion (e.g., rate of rotation) of the image capture device 300. Othertypes of movement sensors 340 may also be present or associated with theimage capture device 300.

The power components 350 may receive, store, and/or provide power foroperating the image capture device 300. The power components 350 mayinclude a battery interface 352 and a battery 354. The battery interface352 operatively couples to the battery 354, for example, with conductivecontacts to transfer power from the battery 354 to the other electroniccomponents of the image capture device 300. The power components 350 mayalso include an external interface 356, and the power components 350may, via the external interface 356, receive power from an externalsource, such as a wall plug or external battery, for operating the imagecapture device 300 and/or charging the battery 354 of the image capturedevice 300. In some implementations, the external interface 356 may bethe I/O interface 332. In such an implementation, the I/O interface 332may enable the power components 350 to receive power from an externalsource over a wired data interface component (e.g., a USB type-C cable).

The user interface components 360 may allow the user to interact withthe image capture device 300, for example, providing outputs to the userand receiving inputs from the user. The user interface components 360may include visual output components 362 to visually communicateinformation and/or present captured images to the user. The visualoutput components 362 may include one or more lights 364 and/or moredisplays 366. The display(s) 366 may be configured as a touch screenthat receives inputs from the user. The user interface components 360may also include one or more speakers 368. The speaker(s) 368 canfunction as an audio output component that audibly communicatesinformation and/or presents recorded audio to the user. The userinterface components 360 may also include one or more physical inputinterfaces 370 that are physically manipulated by the user to provideinput to the image capture device 300. The physical input interfaces 370may, for example, be configured as buttons, toggles, or switches. Theuser interface components 360 may also be considered to include themicrophone(s) 314, as indicated in dotted line, and the microphone(s)314 may function to receive audio inputs from the user, such as voicecommands.

The image capture device 300 may be used to implement some or all of themethods described in this disclosure, such as the method 400 describedin FIG. 4 .

FIG. 4 is a flow diagram of an example of a method 400 for flarecompensation in accordance with embodiments of this disclosure. As shownin FIG. 4 , the method 400 includes obtaining 410 an image via an imagesensor. The method 400 includes generating 420 a grid on the image. Theintersection of the lines of the grid may be referred to as vertices.The lines of the grid may be used to partition the image into tiles(e.g., blocks), and each corner of a tile corresponds to a vertex of thegrid. Accordingly, each tile comprises 4 vertices. Adjacent tiles sharetwo vertices. The image may comprise any number of tiles, and the tilesmay be of any size. For example, each tile of the image may be 4pixels×4 pixels, 16 pixels×16 pixels, 32 pixels×32 pixels, 64 pixels×64pixels, or any other suitable dimension. The tiles are not limited tosquare tiles and may be of any shape and have any number of vertices.For example, tiles may be triangular, hexagonal, octagonal, or any othershape or size. In some embodiments, the image may be partitioned intomultiple tile shapes, sizes, or both.

The method 400 includes determining 430 a flare level of each vertex ofthe grid. The flare level may be determined using any flare compensationalgorithm. The determined flare level may be a level of flare that is tobe suppressed, and it may be a field dependent value to subtract fromthe pixel values. The flare level may correspond to the amount of flarecompensation to be applied to a tile.

The method 400 includes assigning 440 each tile a maximum flare levelvalue. The maximum flare level value assigned to a tile may be the flarevalue of the vertex of that tile that has the highest value. In anexample where the flare value of the first vertex of a tile is 10, theflare value of the second vertex of the tile is 7, the flare value ofthe third vertex of the tile is 8, and the flare value of the fourthvertex of the tile is 5, the tile may be assigned a flare value of 10since 10 is the highest flare value of the 4 vertices.

The method 400 includes sorting 450 the tiles. The sorting 450 of thetiles may include ranking each tile by the amplitude of correctionneeded. The tiles may be sorted according to their respective maximumflare levels. For example, the tiles may be sorted in descending orderfrom the tile with the highest maximum flare level to the tile with thelowest maximum flare level.

The method 400 includes applying 460 flare compensation to a subset ofthe tiles to obtain a processed image. The flare compensation may beapplied using linear interpolation. The processed image may be stored ina memory, transmitted to another device, displayed on one or moredisplays, or any combination thereof. The flare compensation applied tothe subset of tiles may be a level of flare to be subtracted from thepixel values. The subset of tiles may be selected based on a percentage.For example, the subset of tiles may be 25% of the total number oftiles. In other examples, the subset of tiles may be 30%, 40%, 50%, orany other percentage. The percentage may be determined based on theamount of flare detected in the image.

Since the flare compensation is only applied to a subset of the tiles,the tiles that are not included in the subset remain unprocessed (i.e.,non-compensated) with respect to flare. Accordingly, a continuityartifact may be formed at the boundary of a processed tile and anon-processed tile. In an example to avoid producing a continuityartifact, the flare value at the boundary of the processed tile and thenon-processed tile may be forced to zero such that no flare compensationis performed at the boundary to ensure pixel value continuity betweenthe two tiles. For example, the flare value may gradually be forced tozero as the boundary between the processed tile and the non-processedtile approaches such that no flare compensation is performed at theboundary to ensure pixel value continuity between the two tiles.

FIG. 5A is a block diagram of a tiled image 500 in accordance withembodiments of this disclosure. An image is partitioned using a grid toobtain the tiled image 500. The intersection of the lines of the gridmay be referred to as vertices. The lines of the grid may be used topartition the image into tiles (e.g., blocks), and each corner of a tilecorresponds to a vertex of the grid. Accordingly, each tile comprises 4vertices. The tiled image 500 may include any number of tiles, forexample 16 tiles, as shown in FIG. 5A. The tiles are shown as squaretiles for simplicity and may be of any shape and have any number ofvertices. For example, tiles may be triangular, hexagonal, octagonal, orany other shape or size. In some embodiments, the image may bepartitioned into multiple tile shapes, sizes, or both.

As shown in FIG. 5A, tile 510 includes a first vertex 520A, a secondvertex 520B, a third vertex 520C, and a fourth vertex 520D. The firstvertex 520A corresponds to the top-left corner of the tile 510. Thesecond vertex 520B corresponds to the top-right corner of the tile 510.The third vertex 520C corresponds to the bottom-right corner of the tile510. The fourth vertex 520D corresponds to the bottom-left corner of thetile 510. As shown in FIG. 5A, the first vertex 520A is shared with thetop-right corner of tile 530, and the fourth vertex 520D is shared withthe bottom-right corner of tile 530.

In this example, the first vertex 520A may have a flare value of 10, thesecond vertex 520B may have a flare value of 7, the third vertex 520Cmay have a flare value of 8, and the fourth vertex 520D may have a flarevalue of 5. Based on the respective flare values of the first vertex520A, the second vertex 520B, the third vertex 520C, and the fourthvertex 520D, the tile 510 may be assigned a flare value of 10 since 10is the highest flare value of the 4 vertices. In this example, the flarevalue of the top-left corner of tile 530 (i.e., vertex 520E) may be 100.Accordingly, tile 530 is shown to be assigned a flare value of 100 inthis example.

FIG. 5B is a block diagram of the tiled image 500 of FIG. 5A showingselected tiles for flare compensation. The flare values for each tileare shown in the circles of the respective tile. As shown in FIG. 5B,tile 530 has a flare value of 100, tile 540 has a flare value of 100,tile 550 has a flare value of 50, and tile 560 has a flare value of 20.

Since flare is typically generated by a localized high-power lightsource (e.g., the sun), most of the time, only a small part of the imagemay be affected by a flare artifact. In many cases, flare artifacts mayaffect less than 25% of an image. Accordingly, it would be inefficientand costly to process all the pixels of the image when less than 25% ofthe image is affected by the flare artifacts. In accordance withembodiments of this disclosure, a subset of tiles are selected toincrease processing speed and efficiency. In this example, the top 25%of the highest flare value of tiles are selected as the subset of tilesfor flare compensation processing. As shown in FIG. 5B, tile 530, tile540, tile 550, and tile 560 have the highest flare values ranging from20 to 100 and are selected as the subset of the total tiles of the imagefor flare compensation.

As shown in FIG. 5B, flare compensation is applied to tile 530, tile540, tile 550, and tile 560 to obtain a processed image. The flarecompensation may be applied using linear interpolation from vertex tovertex. The flare compensation applied to this subset of tiles may be alevel of flare to be subtracted from the pixel values. Since the flarecompensation is only applied to a subset of the tiles, the tiles thatare not included in the subset remain unprocessed (i.e.,non-compensated) with respect to flare. The unprocessed tiles in thisexample are shown as non-shaded tiles in FIG. 5B. Accordingly, acontinuity artifact may be formed at the boundary of a processed tile,such as tile 530, and a non-processed tile, such as tile 510. To avoidproducing a continuity artifact, the flare value at the boundary of thetile 530 and the tile 510 may be forced to zero such that no flarecompensation is performed at the boundary to ensure pixel valuecontinuity between the two tiles. For example, the flare value maygradually be forced to zero as the boundary between the processed tileand the non-processed tile approaches such that no flare compensation isperformed at the boundary to ensure pixel value continuity between thetwo tiles.

Local tone mapping (LTM) may be used to raise the contrast where it hasbeen lowered during global tone mapping (GTM) processing. GTM processingmay include applying a look up table (LUT) on each pixel value. GTMprocessing may decrease the contrast for pixels that have a value at alevel for which the LUT has a slope less than 1. Accordingly, contrastcompensation would be needed in this example.

FIG. 6 is a flow diagram of an example of a method 600 for contrastcompensation in accordance with embodiments of this disclosure. As shownin FIG. 6 , the method 600 includes obtaining 610 an image via an imagesensor. The method 600 includes determining 620 a thumbnail image basedon the image. The thumbnail image may be based on a grid. Theintersection of the lines of the grid may be referred to as vertices.The lines of the grid may be used to partition the image into thumbnailtiles (e.g., blocks), and each corner of a thumbnail tile corresponds toa vertex of the grid. Accordingly, each tile comprises 4 vertices.Adjacent tiles share two vertices. The image may comprise any number ofthumbnail tiles, and the thumbnail tiles may be of any size. Forexample, each thumbnail tile of the image may be 4 pixels×4 pixels, 16pixels×16 pixels, 32 pixels×32 pixels, 64 pixels×64 pixels, or any othersuitable dimension. The tiles are not limited to square tiles and may beof any shape and have any number of vertices. For example, tiles may betriangular, hexagonal, octagonal, or any other shape or size. In someembodiments, the image may be partitioned into multiple tile shapes,sizes, or both.

The method 600 includes determining 630 a contrast value of eachthumbnail tile using the vertices of the grid. The contrast value may bedetermined using any contrast compensation algorithm. The determinedcontrast value may be a level of contrast that is to be suppressed orenhanced, and it may be a field dependent value to add or subtract fromthe pixel values. The contrast value may correspond to the amount ofcontrast compensation to be applied to a tile.

Determining 630 the contrast value of each thumbnail tile may includeassigning each tile a maximum contrast value. The maximum contrast valueassigned to a tile may be the contrast value of the vertex of that tilethat has the highest value. In an example where the contrast value ofthe first vertex of a tile is 10, the contrast value of the secondvertex of the tile is 7, the contrast value of the third vertex of thetile is 8, and the contrast value of the fourth vertex of the tile is 5,the tile may be assigned a flare value of 10 since 10 is the highestflare value of the 4 vertices.

The method 600 includes sorting 640 the tiles. The sorting 640 of thetiles may include ranking each tile by the amplitude of correctionneeded. The tiles may be sorted according to their respective maximumcontrast levels. For example, the tiles may be sorted in ascending orderfrom the tile with the lowest maximum contrast value to the tile withthe highest maximum contrast value.

The method 600 includes applying 650 contrast compensation to a subsetof the tiles to obtain a processed image. The contrast compensation maybe applied using linear interpolation. The processed image may be storedin a memory, transmitted to another device, displayed on one or moredisplays, or any combination thereof. The contrast compensation appliedto the subset of tiles may be a level of contrast to be added orsubtracted from the pixel values. The subset of tiles may be selectedbased on a percentage. For example, the subset of tiles may be 25% ofthe total number of tiles. In other examples, the subset of tiles may be30%, 40%, 50%, or any other percentage.

Since the contrast compensation is only applied to a subset of thetiles, the tiles that are not included in the subset remain unprocessed(i.e., non-compensated) with respect to contrast correction.Accordingly, a continuity artifact may be formed at the boundary of aprocessed tile and a non-processed tile. In an example to avoidproducing a continuity artifact, the contrast value at the boundary ofthe processed tile and the non-processed tile may be forced to zero suchthat no contrast compensation is performed at the boundary to ensurepixel value continuity between the two tiles. For example, the contrastcompensation value may gradually be forced to zero as the boundarybetween the processed tile and the non-processed tile approaches suchthat no contrast compensation is performed at the boundary to ensurepixel value continuity between the two tiles.

The implementations described herein may be applied to blue-fringingcorrection. Blue-fringing occurs around saturated values and may appeararound the sky around a tree. The implementations described herein maybe adapted to determine regions of the image that may most benefit fromblue-fringing correction. The blue-fringing correction may be determinedfrom statistics used for an auto-exposure (AE) algorithm. The AEalgorithm may include a count of saturated values per region or tile.

While the disclosure has been described in connection with certainembodiments, it is to be understood that the disclosure is not to belimited to the disclosed embodiments but, on the contrary, is intendedto cover various modifications and equivalent arrangements includedwithin the scope of the appended claims, which scope is to be accordedthe broadest interpretation so as to encompass all such modificationsand equivalent structures as is permitted under the law.

What is claimed is:
 1. An image capture device comprising: an imagesensor configured to obtain an image; a processor configured to:generate a grid on the image, forming tiles; determine a fringing levelof each vertex of vertices of the tiles; sort all of the tiles based onthe fringing level of each tile so that the tiles are sorted in adescending order from the tile with a highest of the fringing levels tothe tile with a lowest of the fringing levels; and apply a fringingcompensation to a subset of the sorted tiles to obtain a processedimage; and a memory configured to store the processed image.
 2. Theimage capture device of claim 1, wherein each vertex corresponds to acorner of at least one tile.
 3. The image capture device of claim 1,wherein each tile comprises four vertices.
 4. The image capture deviceof claim 3, wherein a maximum fringing level for each tile correspondswith a highest fringing level of the four vertices.
 5. The image capturedevice of claim 1, wherein the subset of the sorted tiles is based on apercentage of a total number of tiles, and the percentage is 25 percent.6. The image capture device of claim 1, wherein the fringing level is ablue-fringing correction.
 7. The image capture device of claim 1,wherein the processor is configured to apply the fringe compensationusing a linear interpolation.
 8. The image capture device of claim 7,wherein the processor is configured to force a zero fringingcompensation at an edge between a first tile and a second tile.
 9. Theimage capture device of claim 8, wherein the first tile is included inthe subset of the sorted tiles and the second tile is excluded from thesubset of the sorted tiles.
 10. The image capture device of claim 1,wherein each tile is 32 pixels×32 pixels.
 11. A method comprising:obtaining an image; generating a grid on the image, wherein the gridcomprises vertices to form tiles; determining fringing levels for thevertices; applying a fringing compensation to a subset of the tilesbased on the fringing levels to obtain a processed image, whereinapplying the fringing compensation includes forcing a zero-fringingcompensation at an edge between a first tile and a second tile of thetiles.
 12. The method of claim 11 further comprising: assign a maximumfringe level for each tile.
 13. The method of claim 11 furthercomprising: sorting the tiles based on a maximum fringe level of eachtile.
 14. The method of claim 11, wherein the subset of tiles is basedon a percentage of a total number of tiles.
 15. The method of claim 11,wherein the fringing is blue fringing.
 16. The method of claim 15,wherein the first tile is included in the subset of tiles and the secondtile is excluded from the subset of tiles.
 17. An image capture devicecomprising: an image sensor configured to obtain an image; a processorconfigured to: determine a thumbnail image based on the image, whereinthe thumbnail image comprises thumbnail tiles; determine a saturationvalue of each thumbnail tile; sort all of the thumbnail tiles based onthe saturation value of each thumbnail tile in an ascending order fromthe thumbnail tile with a lowest saturation value to the thumbnail tilewith a highest saturation value; and apply a correction to a subset ofthe sorted thumbnail tiles to obtain a processed image; and a memoryconfigured to store the processed image.
 18. The image capture device ofclaim 17, wherein the processor is further configured to force a zerocompensation value at an edge between a first tile and a second tile.19. The image capture device of claim 18, wherein the first tile isincluded in the subset of the sorted thumbnail tiles and the second tileis excluded from the subset of the sorted thumbnail tiles.
 20. The imagecapture device of claim 17, wherein the processor is further configuredto count saturated values per tile.